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FINAL  REPORT 


GRANT  NUMBER:  F4962000 10336 

TITLE:  Metareasoning  for  More  Effective  Human-Computer  Dialogue 
PRINCIPAL  INVESTIGATORS: 

Don  Perlis  and  Mike  Anderson 
University  of  Maryland 
College  Park,  MD  20742 


OBJECTIVES: 

This  study  will  foster  new  understanding  of  error-repair  in 
intelligent  systems,  of  very  broad  applicability.  Such  repair  is 
necessary  where  there  is  complex  and  noisy  data,  as  is  prevalent  in 
military  settings . 


STATUS  OF  EFFORT: 

See  Accomplishments,  below.  The  grant  period  ended  December,  2002. 


SUMMARY: 

The  research  project  explores  specific  meta-dialogue  behaviors  in 
terms  of  both  how  a  system  could  be  made  to  perform  them,  and  to  what 
extent  they  can  increase  overall  system  performance.  We  focus  on 
two  types  of  metadialogue  capabilities:  ability  to  detect  and  recover 
from  anomalous  dialogue  patterns  in  simple  exchanges,  and  on-line 
extensions  or  changes  to  working  vocabulary. 

Our  main  method  involves  detailed  representation  of  the  dialogue 
context,  separating  domain,  language,  and  dialogue  specific  aspects. 
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and  significant  amounts  of  meta-reasoning  about  the  system's 
processing  of  these  representations.  An  existing  logical  inference 
system,  ALMA/CARNE,  developed  as  part  of  a  pilot  study,  is  being  used 
in  an  implementation  phase  of  this  work.  We  are  also  engaged  in  a 
study  of  existing  dialogue  corpora  to  investigate  the  range  and 
frequency  of  meta-dialogue  expressions  in  different  task  domains. 


ACCOMPLISHMENTS : 

1.  We  designed  and  implemented  extensions  of  the  existing  active  logic 
engine  and  dialogue  manager,  to  be  able  to  engage  in  a  variety  of 
action  directive  subdialogues,  which  include: 

(a)  a  request  for  action  performance  by  the  user 
Cb)  performance  of  an  action  by  the  system 

(c)  feedback  ("followup")  from  the  user  (which  might  be  positive 
indicating  acceptance  of  the  action,  or  negative,  possibly 
Coupled  with  a  replacement  action) . 

2.  We  also  designed  a  more  powerful  architecture  that  will  allow  us  to 
process  new  words  and  phrases  and  implicit  and  explicit  quotation, 

in  order  to  adapt  to  a  variety  of  users  and  contexts.  As  a 
result,  users  will  be  able  to  instruct  the  system  in  exchanges 
such  as: 

(d)  I  use  "transportation"  to  refer  to  any  vehicle,  not  just  a  car. 

(e)  Peking  is  the  old  name  for  Beijing. 

3.  Our  most  recent  work  has  allowed  us  to  implement  a  system  (ALFRED: 
Active  Logic  for  Reason  Enhanced  Dialog)  that  can,  in  simple  cases, 

(i)  recognize  the  need  to  for  meaning  clarification,  (ii)  generate  a 
request  for  such  clarification,  and  (iii)  process  a  direct  supplying 
of  that  clarification,  as  in: 

(f)  A:  Shoot  the  Chicago  train  to  Boston. 

B:  What  does  "shoot"  mean? 

A:  Shoot  means  send. 

B:  OK  [and  sends  the  Chicago  train  to  Boston] 
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This  system  was  demonstrated  at  the  University  of  Maryland  Research 

Review  Day  (March  2003) .  A  more  advanced  version  is  slated  for  demonstration 

at  IJCAI  2003. 

Implementing  the  more  powerful  architecture  above  involved  integrating 
a  new  parser  into  the  system,  as  well  as  recoding  several  other 
modules.  We  are  now  begining  to  extend  the  system,  allowing  it  not 
only  to  learn  new  words  and  phrases  as  in  (d-e-f)  above,  but  also 
learn  novel  concepts. 

In  addition,  isolating  the  metareasoning  components  involved  in 
understanding  language  from  the  other  parts  of  the  system  allows  the 
dialog  system  to  be  easily  oriented  to  different  domains. 

4.  These  capabilities  advance  the  state  of  the  art  for  current 
natural  language  dialogue  systems,  and  represent  a  major  step  toward 
our  long-term  goal  of  "conversational  adequacy"  as  described  in  our 
earlier  papers. 

5.  We  have  also  recently  begun  a  corpus  study  of  different  kinds  of 
meta-language,  their  form,  frequency,  and  effect.  Results  from  this 
study  will  help  to  implement  more  sophisticated  reasoning  mechanisms  for 
dealing  with  the  various  kinds  of  misunderstanding  in  dialog. 

6.  Summary  of  Dissertation. 

Khemdut  Purang,  Systems  that  Detect  and  Repair  their  own  Mistakes’', 
University  of  Maryland  2001 . 

Making  mistakes  is  an  inescapable  aspect  of  everyday  life.  We 
constantly  make  mistakes,  recognize  them  and  try  to  correct 
them.  Mistakes  are  inevitable  because  of  the  incompleteness  of  our 
knowledge  of  the  world,  its  inherent  uncertainty  and  its  being  in  a 
constant  state  of  change.  We  can  never  know  for  sure  that  what  we  know 
is  true  and  the  actions  that  we  take  based  on  these  beliefs  can 
therefore  be  misguided.  Sooner  or  later  we  act  based  on  some  false 
belief  or  the  world  changes  in  an  unexpected  way  and  we  fail  to 
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achieve  our  goal.  But  the  fact  that  we  can  recognize  and  repair  these 
errors  mitigates  their  effects .  Software  systems  face  the  same 
problems.  The  difference  is  that  they  do  not  ususally  have  as  robust  a 
capability  as  we  have  to  detect  and  respond  to  their  mistakes.  This  is 
part  of  what  makes  them  seem  brittle  and  user-unfriendly.  This  problem 
is  not  likely  to  get  any  better  as  the  systems  exhibit  more  complex 
behaviors  in  more  realistic  domains.  Our  work  begins  to  address  that 
problem  by  focusing  on  the  computational  capabilities  required  of 
software  systems  for  them  to  be  able  to  autonomously  recognize  and 
respond  to  their  own  mistakes.  We  study  in  particular,  mistaken 
beliefs,  intentions  and  actions  in  agents  that  have  some  goals  to 
achieve.  Intuitively  enough,  the  basic  capabilities  required  are  an 
ability  to  inspect  their  past  behavior  and  computations  and  the  past 
states  of  the  world  and  to  use  that  to  determine  their  future 
behavior.  These  abilities  are  not  typically  available  in  software 
systems.  We  have  implemented  a  general  logical  framework  in  which  one 
can  specify  the  behavior  of  an  agent  that  supports  this  kind  of 
representation  and  computation  .  We  have  implemented  agents  that 
detect  and  respond  appropriately  to  their  mistakes  in  some  aspects  of 
language  processing.  We  have  also  implemented  a  system  that  handles 
its  mistaken  beliefs  in  any  domain  that  can  be  described  using  the 
language  of  non-monotonic  logic.  This  system  was  tested  on  a  test 
suite  that  we  compiled  from  examples  of  non-monotonic  reasoning  in  the 
literature.  We  finally  provide  a  design  of  the  representations  and 
algorithms  for  handling  mistakes  in  an  agent  that  acts  in  the  world 
and  has  mistaken  beliefs,  intentions  and  actions.  Implementing  such  an 
agent  is  the  next  step  in  this  work. 
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1.  Representations  of  Dialogue  Statue  for  Domain  and  Task  Independent 
Meta-Dialogue.  David  Traum,  Carl  Andersen,  Yuan  Chong,  Darsana  Josyula, 
Yoshi  Okamoto,  Khemdut  Purang,  Michael  0 ’ Donovan-Anderson  and  Don  Perlis 
Electronic  Transactions  on  AI  6,  2002. 

2.  The  Use-Mention  Distinction  and  its  importance  to  HCI.  Michael 

L.  Anderson,  Yoshi  Okamoto,  Darsana  Josyula,  and  Don  Perlis.  Proceedings 
of  the  Sixth  Workshop  on  the  Semantics  and  Pragmatics  of  Dialog.  2002. 

3.  Meta-reasoning  for  Conversational  Adequacy.  Michael  L.  Anderson 
Invited  talk,  Deutsches  Forschungszentrum  fr  Knstliche  Intelligenz,  2002 

4.  Seven  Days  in  the  Life  of  a  Robotic  Agent.  Waiyian  Chong,  Mike 
Q!Donovan-Anderson,  Yoshi  Okamoto  and  Don  Perlis. 

First  GSFC/JPL  Workshop  on  Radical  Agent  Concepts, 

2001,  NASA  Goddard  Space  Flight  Center,  Greenbelt,  MD,  USA 
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5.  Meta-reasoning  for  Intelligent  Dialog  Repair,  Khemdut  Purang,  David  Traum, 
Darsana  V,  Purushothaman ,  Waiyian  Chong,  Yoshi  Okamoto  and  Don  Perlis, 
November  2000. 

6.  Time-Situated  Agency:  Active  Logic  and  Intention  Formation. 

Cognitive  Agents  Workshop,  2002.  With  M.  Anderson, 

D.  Josyula,  Y.  Okamoto. 

7.  Handling  Uncertainty  with  Active  Logic,  Man jit  Bhatia,  Paul  Chi,  Waiyian 
Chong,  Darsana  P.  Josyula,  Yoshi  Okamoto,  Don  Perlis  and  Khemdut  Purang. 
March  2001.  To  be  presented  at  the  2001  AAAI  Fall  Symposium  Series  in  North 
Falmouth,  MA. 

8.  Zi  at  the  Interface  and  Beyond,  Yoshi  Okamoto. 

Presented  at  the  L0T-NWCL  Summer  School.  Utrecht,  the  Netherlands: 
Universiteit  Utrecht,  2001. 

9.  Feature  Analysis  of  Zibun,  Yoshi  Okamoto.  April  2000.  Presented  at  45th 
Annual  Conference  of  the  International  Linguistic  Association.  Washington, 
DC;  Georgetown  University. 

10.  Split  Antecedents  <E0>  la  Movement,  Yoshi  Okamoto.  July 

2000.  Snippets,  Issue  2.  Milano,  Italy:  LED  -  Edizioni  Universitarie 
di  Lettere  Economia  Diritto. 

11.  Agents  that  recognize  and  repair  their  own  mistakes.  PhD 
dissertation.  Khemdut  Purang.  August  2001. 

12.  D.  Perlis.  What  does  it  take  to  refer?  (Invited  paper.) 

J.  of  Consciousness  Studies,  2000. 

13.  Symbol  Systems.  With  Michael  L.  Anderson. 

Encyclopedia  of  Cognitive  Science.  2002. 

14.  Invited  one-hour  talk  and  paper  on  self -reference,  presented 
at  PhiLog,  Copenhagen,  October  2002. 
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INTERACTIONS : 

1.  invited  talk  at  New  Mexico  University 

2.  contact  and  discussions  with  ARL  (Dr.  John  Gurney) 

3.  invited  talk  at  PhiLog  (Logic  Colloquium  in  Denmark) 

4.  invited  talk  at  DFKI . 

Inventions :  none 
Awards:  none 
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